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-- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

• If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1)IE Responsive to communication(s) filed on 02 October 2003 . 
2a)D This action is FINAL. 2b)S This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) I3 Claim(s) 7-77 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) M Claim(s) 7-77 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

11) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 
Priority under 35 U.S.C. §§119 and 120 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (0- 

a)QAII b)D Some*c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

13) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application) 

since a specific reference was included in the first sentence of the specification or in an Application Data Sheet. 
37 CFR 1.78. 

a) □ The translation of the foreign language provisional application has been received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 since a specific 

reference was included in the first sentence of the specification or in an Application Data Sheet. 37 CFR 1.78. 
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DETAILED ACTION 
Claim Rejections - 35 USC § 102 

1 . The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1 ) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

2. Claims 1, is rejected under 35 U.S.C. 102(b) as being anticipated by Corey 
et al (5,703,655) 

As to claim 1, note the Corey et al reference figures 1 and 2, disclose system 
and method for indexing and retrieving portions of video programming data contained in 
a video/audio storage area of stored video programs which includes broadcast 
television and further disclose a method comprising steps of: the claimed "indexing a 
media collection..." is met by Control Module (CM) 60 (figs. 1, 2, col. 3, lines 37-51 and 
col. 4, lines 17-44), note that CM 60 controls the receiver-tuner 24 by supplying it with 
control signals and also compression/decompression module 36 to create various index 
records based on the content of the video/audio data "the media collection" and store in 
video/audio storage device 40, the video retrieval system 20, the claimed "searching the 
indexed library to identify a set of candidate program segments based on a search 
criteria..." is met by User Input Device (UID) 76 (col. 4, line 39-col. 5, line 4), note that 
UID 76 allows the user to search the storage devices 40 and 72, "indexed library," to 
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identify one or more desired video segments, "set of candidate program segments," 
"based on the search criteria" (col. 5, lines 31-40), note that closed caption text, which is 
a video/audio content, is used as search criteria; and Television Monitor 84, is use for 
browsing the set of program segments to select a segment for viewing, note that the 
system allows capturing of desire video programs (figs. 6, 12A, col. 5, lines 5-57 and 
col. 6, line 51 -col. 7, line 40). 

3. Claims 8-13, are rejected under 35 U.S.C. 102(b) as being anticipated by 
Fasciano (6,336,093). 

As to claim 8, note the Fasciano reference figures 6 and 7, discloses apparatus 
and method using speech recognition and scripts to capture author and playback 
synchronized audio and video and further discloses method of indexing media for 
browsing, comprising the following: the claimed indexing a media collection according to 
detection of speaker voice characteristics" is met by Speech Recognition Module (SRM) 
70 (fig. 7, col. 6, line 63-col. 7, line 17 and lines 40-53), note that SRM 70 detects sound 
patterns of speech, example each Journalist, Editor, etc., (col. 7, lines 41-47) within a 
received video, capturing and indexing the video or news information "media" according 
to the voice of the Journalist or Editor "speaker;" the claimed "receiving a search query 
from a user to locate a media segment from the indexed media collection" is met by the 
Graphical User Interface (GUI) 40 (fig. 1 and col. 3, lines 1 1-44), note the User Input 44 
is used to select video segments and sound pattern are use in performing a search 
within SRM 70, where the user is presented with a portion of the indexed video 
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collection according to the search query (fig. 6, col. 4, lines 7-16, col. 5 , line 47-col. 6, 
line 15 and line 54-63). 

As to claim 9, Fasciano further disclose indexing a video collection according to 
visual information (col. 6, lines 8-15 and lines 54-63). 

As to claim 10-13, Fasciano further discloses identifying speaker speech 
segments (col. 7, lines 40-54) note that the sound patterns of each Journalist or Editor 
is selected to identify a speaker speech segments, and the identify speaker segments is 
used to extract video segments and summaries of the video segments of an individual 
(figs. 4-6 and col. 6, lines 21 +), note further the User GUI presents the extracted 
summaries in response the user search query. 



4. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, rf the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 2-4 and 6, are rejected under 35 U.S.C, 103(a) as being unpatentable 
over Corey et al (5,703,655) as applied to claim 1 above, in view of Fasciano 
(6,336,093). 

As to claim 2, Corey further teaches extracting from the media collection for each 
segment of candidate image segments one of closed captioning data as a searchable 
text data (col. 5, lines 31-57), comparing closed captioning text "the search criteria" to 
the searchable text data in the storage device 40 and 72 (col. 7, lines 30-57). 



Claim Rejections - 35 USC § 103 
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Corey fails to explicitly teach extracting speech recognition data when closed 
captioning data is unavailable. 

However, Fasciano teaches using speech recognition and scripts to capture 
author and playback synchronized audio and video where in the absence of closed 
captioning data, where a speech pattern is used as searchable data to retrieve video 
(figs. 6, 7 and col. 6, line 54-col. 7, line 15 and line 16+). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to incorporate the teaching of Fasciano into the system of Corey to 
provide speech recognition system to extract patterns of speeches, capture related 
video, store and playback as desire. 

As to claims 3 and 4, Corey fails to explicitly teach browseable image for each 
segment of the candidate program segments which includes keywords identified in the 
searable text data for display in the browseable image and selecting a display segment 
from the set of candidate program segments and displaying the associated browseable 
image with associated keywords. 

However, Fasciano teaches providing browseable image for each segment of the 
candidate program segments (figs. 4-6, and col. 6, lines 15+) which includes keywords 
(keywords 100) identified in the searable text data for display in the browseable image 
and selecting a display segment from the set of candidate program segments and 
displaying the associated browseable image with associated keywords (col. 6, line 
54-col. 7, line 15) 
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Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to incorporate the teaching of Fasciano into the system of Corey to 
provide a visual presentation to the user of a search result and it's associate keywords, 
to permit the user to view or browse through and select images as desire. 

Claim 6 is met as previously discussed with respect to claim 3 
6. Claim 5 and 7, is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Corey et al (5,703,655) in view of Fasciano (6,336,093) as applied to claims 3 and 6 
above, and further in view of Hoffert et al (5,983,176). 

As Corey as modified by Fasciano fail to explicitly teach searchable text data 
associated with the selected display segment includes a first word having low 
information content and second word having high information content and forming 
browseable image includes selecting the second word as keyword and rejecting the first 
word as a keyword. 

However, note Hoffert et al reference disclose evaluation of media content in a 
media files, searches for the files in the database and displays the search results based 
on the content of the media files where high level content attributes are used for short 
video sequences to enhanced searching (col. 21, lines 10-28, col. 24, lines 15-35 and 
Appendix A), note that text and closed captioning data is also used as searchable text. 

Therefore it would have be obvious to one of ordinary skill in the art at the time of 
the invention to incorporate the teaching of Hoffert into the system of Corey as modified 
by Fasciano to used high content level data to perform a search to reduce error in a 
search result. 
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As to claim 7, Corey as modified by Fasciano, teach Journalist or Editor 
"anchor-person," but fail to explicitly teach an anchor-person associated with the 
selected video and where the anchor-person with low information content and field shot 
image of an event of high information content and forming a browseable image by 
selecting the field shot image as key image and rejecting the image of the anchor- 
person as key image. 

However, note Hoffert et al reference disclose evaluation of media content in a 
media files, searches for the files in the database and displays the search results based 
on the content of the media files where high level content attributes are more meaning 
used for short video sequences to enhanced searching (col. 21, lines 10-28, col. 24, 
lines 15-35 and Appendix A), note that text and closed captioning data is also used as 
searchable text. 

Therefore it would have be obvious to one of ordinary skill in the art at the time of 
the invention to incorporate the teaching of Hoffert into the system of Corey as modified 
by Fasciano to discard Editor with low information content and used high content 
information content to perform a search to reduce error in a search result. 
7. Claims 14-17 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Fasciano (6,336,093) as applied to claim 8 above, and view of Halverson (6,523,061). 

As to claims 14, Fasciano fails to explicitly teach a search query from a user 
further comprising a natural language query. 

However, Halverson teaches system and method for agent-based navigation in a 
speech-based data navigation system where, when a spoken request is received from a 
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user, it is interpreted and the resulting interpretation is used automatically construct an 
operational query (figs. 1, 2, 4 and col. col. 7, line 64-col. 8, line 25). 

Therefore it would have be obvious to one of ordinary skill in the art at the time of 
the invention to incorporate the teaching of Halverson into the system of Fasciano in 
order to an apparatus to interpreted natural language and use the interpretation to 
perform queries to achieve reduction in speech recognition error in grammars. 

As to claims 15 and 16, Halverson further discloses contextual information from a 
previous interaction with the natural language and return an answer in an HTML format 
(fig. 4, col. 10, lines 6-19). 

As to claim 17, Halverson further discloses where the natural language search 
query from a user, comprises generating a semantic description of the natural language 
query in terms of keyword/value pairs (fig. 3 and col. 1 1 , lines 35-col. 12, lines 1 1 ). 



8. Applicant's arguments with respect to claims 1-8 have been considered but are 
moot in view of the new ground(s) of rejection. With respect to claim 1 , Applicant argues 
that Corey teaches video indexing based on closed captioning text and not video 
content. Examiner disagrees since media content, can be broadly interpreted as closed 
captioning text. Furthermore, the claim does not require the use of some other form of 
content for video indexing and further Applicant, admits in claim 2 of using closed 
captioning text to extract media information. For the above reasons, the rejection of 
claim 1 is maintained. This Office Action is Non-Final. 



Response to Arguments 



• 



* 
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Conclusion 



9. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

McGee et al (6,496,228) disclose significant scene detection and frame filtering 
for a visual indexing system using dynamic thresholds. 

Altunbasak et al (6,389,168) disclose object-based parsing and indexing of 
compressed video streams. 

Dimitrova et al (6,185,363) disclose visual indexing system. 

Yeo et al (5,821,945) disclose method and apparatus for video browsing based 
on content and structure. 

10. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Annan Q Shang whose telephone number is 703-305- 
2156. The examiner can normally be reached on 700am-500pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John W Miller can be reached on 703-305-4795. The fax phone number 
for the organization where this application or proceeding is assigned is 703-746-5991. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the Customer Service whose telephone number is 
703-306-0377. 





Annan Q. Shang 



^ JOHN MILLER 
SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 2600 



